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(54) Detection and rendering of text in halftone tinted areas 



(57) A method and apparatus improves digital re- 
production of a compound document image containing 
half-tone tint regions and text and/or graphics embed- 
ded within the half-tone tint regions. The method entails 
determining (30) a local average pixel value for each pix- 
el in the image, then discriminating and classifying (40) 
based on the local average pixel values, text-graphics 
pixels from half-tone tint pixels. Discrimination can be 
effected by calculating a range of local averages within 
a neighborhood surrounding each pixel; by calculating 
edge gradients based on the local average pixel values; 
or by approximating second derivatives of the Ic^l av- 
erage pixel values based on the local average^/> fext/ 
graphics pixels are rendered using a rendering method 
appropriate for that type of pixel; half-tone tint pixels are 
rendered using a rendering method appropriate for that 
type of pixel. 
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Description 

In general, the Invention relates to reproduction of compound images containing both text/graphical information 
and half-tone tinted regions. In particular, the invention relates to reproduction of images in which text and/or graphical 
5 information is embedded within a half-tone tinted region. 

When digitally reproducing (e.g. by means of a digital copier) compound documents containing tinted regions 
produced by a half-toning process and text and/or graphics embedded within the tinted regions, it is desirable to seg- 
ment the image into text/graphical portions and tinted, half-toned portions in order to optimize print quality. (Text and/ 
or graphics will be referred to herein simply as °text°.) Often, text areas are enhanced and then rendered using thresh- 
10 olding or error diffusion. Low frequency half-toned areas can be reproduced using error diffusion; high frequency half- 
toned areas may be reproduced using de-screen filtering followed by half-toning using a screen appropriate for the 
output print device, e.g. the digital copier printer. Methods for doing so are known in the art. See, for example, US- 
A4.1 94,221 and US-A-5,293,430. 

Discrimination between, and separation of the image into, areas of text and half-tone tinting is not always a simple 
is proposition. Often, documents contain text embedded within half-tone tint areas. In previous reproduction methods, 
such embedded text was reproduced in the same manner as the surrounding half-tone tinted region. Hence, it was 
low-pass filtered and re-screened with the surrounding pixels constituting the half-tone tinting. As a result, the embed- 
ded text would have a blurred or "ragged" appearance (Fig. 1 ). 

The present invention overcomes this problem by detecting text that is embedded within half-tone tint regions and 
20 rendering the text and tint regions separately by processes appropriate for each type of pixel. Thus, in a first aspect, 
the invention provides a method for reproducing a compound document image containing a half-tone tint region with 
text and/or graphics embedded in the half-tone tint region. In the method, a digital image of the document is provided. 
The digital image includes a multiplicity of pixels, each of which has a value that represents the tone of the pixel. An 
"average image 0 of the document is de^ pixel locatiohV an average of the values of the 

25 pixels within an "averaging window" encompassing that pixel location. A "discrimination value" is then determined for 
each pixel location indicating whether the pixel is a half-tone tint pixel or a text/graphics pixel. This information is then 
used to control a print device to reproduce at least a portion of the document image. 

Embodiments of the inventive method can include the following features. The average image can be calculated 
using a simple average, or it can be calculated using a weighted average. The averaging window used to calculate 
30 the average image can have predetermined dimensions; it can also have variable dimensions which vary based on 
scanning resolution used to generate the digital image or frequency components of the digital image. 

The discrimination value can be based on the average image, either by calculating a range of the average pixel 
values within a neighborhood surrounding each pixel, by calculating average value edge gradients for each pixel lo- 
cation, or by approximating second derivatives of the average pixel values for each pixel location. 
35 The method preferably further comprises analyzing said digital image to determine, for each said pixel, whether 

said pixel is in a high frequency half-tone tint region or a low frequency half-tone tint region. 

The method preferably further comprises controlling said print device in accordance with said determination as to 
whether each said pixel is in a high frequency half-tone tint region or a bw frequency half-tone tint region to output a 
reproduction of at least a portion of said document image. The method preferably further comprises de-screen filtering 
40 said digital image before outputting said reproduction of at least a portion of said document image. 

Preferably said portion consists of the embedded text and/or graphics and wherein said outputting comprises 
printing said embedded text and/or graphics without printing the half-tone tint region in which said text and/or graphics 
is embedded. 

The method allows the embedded text and/or graphics to be "extracted" so that the embedded text and/or graphics 

45 can be printed without the half-tone tinting in which it is embedded. 

In another aspect, the invention provides an apparatus for digitally reproducing a compound document image 
containing half-tone tint regions with text and/or graphics embedded in the half-tone tint regions. The apparatus includes 
an input device that allows a digital image of the document to be input. The digital image includes a multiplicity of pixels, 
each of which has a value to represent the tone of tne pixel. The apparatus has an averaging device which determines 

so an average image of the document by calculating, for each pixel location, an average of pixel values within an averaging 
window that encompasses that pixel location. The apparatus also has a text detector that determines a discrimination 
value for each pixel indicating whether that pixel is a half-tone tint pixel or a text/graphics pixel, and a video processor 
that controls a print device in accordance with said discrimination values to output a reproduction of at least a portion 
of the document image. 

55 Embodiments of the inventive apparatus can include the following features. The apparatus preferably further com- 

prises a frequency detector which analyzes said digital image to determine for each said pixel whether said pixel is in 
a high frequency half-tone tint region or a low frequency half-tone tint region, and wherein said video processor further 
controls said print device in accordance with said determination as to whether each said pixel is in a high frequency 
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half-tone tint region or a lowf requency half-tone tint region to outputa reproduction of at least a portion of said document 
image. 

The apparatus preferably further comprises a de-screening filter which removes screen effects from said digital 
image before said print device outputs said reproduction of at least a portion of said document image. 
s The input device may comprise a scanner. 

The apparatus may comprise a digital copier. 

Embodiments of the invention will now be described, by way of example, with reference to the accompanying 
drawings, in which: 

10 Fig. 1 shows an example of text embedded within half-tone tinted areas as rendered by a prior art reproduction 
method; 

Fig. 2 is a flowchart showing the steps of an embodiment of a digital reproduction method according to the invention; 
Fig. 3 is a block diagrammatic representation of a reproducing apparatus which performs the method shown in 
Fig. 2; 

15 Figs. 4A and 4B are graphs showing variation of pixel values, in the horizontal direction, for a vertical line embedded 

in a half-tone tinted region before and after averaging, respectively; and 

Fig. 5 shows an example of text embedded within half-tone tinted areas as rendered according to the invention. 

As shown in Fig. 2, image reproduction according to the invention begins with input 10 of a digital representation 

20 of an image. The digital representation includes a multiplicity of pixels, and the value of each pixel indicates the tone 
of the pixel. In an eight-bit representation, for example, a pixel value of 255 typically indicates a totally white pixel, and 
a pixel value of zero indicates a totally black pixel. As shown in Fig. 3, digital copier 100 includes a scanner 102 which 
scans a document 1 04 having text and tinted regions and outputs a digitized representation, Vj j of the document image. 
Vj j may be stored in data buffer 106. 

25 The image data Vj j is analyzed, in frequency analysis step 20, using frequency analyzer 1 09. Frequency analyzer 

109, as is known in the art (e.g. US-A-4, 194,221 and US-A-5,293,430, cited above), analyzes image data and deter- 
mines whether it is continuous tone, low frequency half-tone, or high frequency half-tone image data. Frequency an- 
alyzer 109 outputs frequency information a^ j for every pixel indicating whether the pixel is located in a region of low 
frequency half-tone tinting or high frequency half-tone tinting. 

30 The image data Vj j from buffer 106 is also processed by an averaging filter 112 which implements an averaging 

process 30 (Fig. 2) to produce an average image Aj j. The average image Aj j consists of a local average value, for 
every pixel location, of the pixel values for that pixel location and the surrounding pixel locations. In other words, the 
local average value is obtained by averaging the pixel values within a local averaging window surrounding each pixel 
in the image. The averaging process 30 (Fig. 2) effected by the averaging filter 112 (Fig. 3) can be represented math- 

35 ematically as a convolution: 



40 where are the filter coefficients. 

The averaging process "smoothes out" variations in the pixel values in the half-tone tint areas, but leaves distinct 
variations where text exists, even when the text is embedded within a half-tone tinted region. For example, Fig. 4A 
represents the variation, in the horizontal direction, of the pixel values in the neighborhood of a vertical line embedded 
within a half-tone tinted region. The half-tone tinted regions are represented by the portions labelled HT; the vertical 

45 line is represented by the portion labelled L. As shown in Fig. 4B, the half-toned regions are smoothed out to nearly 
uniform values, as shown in the regions labelled HT, while the region L' corresponding to the vertical line still shows 
a significant change in the local pixel value. 

In one implementation of the invention, the size of the averaging window can be predetermined. Alternatively, it is 
preferably for the size of the averaging window to vary based on the horizontal and vertical frequency components of 

50 the input video signal Vj j. At a minimum, the averaging window must be as large, in each of the horizontal or vertical 
directions, as the "wavelength" of the image in the corresponding direction, i.e. the number of pixels over which the 
periodic components of the image repeats themselves. If the averaging window is too large, however, the possibility 
exists for the local average value for a text pixel to be excessively biased by surrounding half-tone tint pixels so that 
it appears as if no text pixel is present at all. Moreover, a larger averaging window is more costly to implement in terms 

55 of required computing power and memory and actual cost. 

For a common scanning resolution of 400 spots per inch (spi) (16 spots/mm), an averaging window size of 3 pixels 
x 3 pixels could be used. An averaging window of this size is able to smooth out 45 degree half-tone tinting at 1 20 lines 
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per inch (Ipi) (5 lines/mm) or higher frequency, without obliterating the existence ot embedded text. For 600 spi (24 
spots/mm) scanning resolution, the averaging window could be expanded to 5 pixels x 5 pixels. For a scanning reso- 
lution of 400 spi (16 spots/mm) in the horizontal direction and 600 spi (24 spots/mm) in the vertical direction, with a 45 
degree half-tone frequency of 120 Ipi (5 lines/mm) or more, an averaging window of 3 pixels horizontally x 5 pixels 
vertically would be appropriate. 

The manner in which the pixel values within the averaging window are averaged can also vary. In one implemen- 
tation, the local average value can be a simple average. In that case, using the 3 pixel by 5 pixel averaging window 
as an example, the filter coefficients C t _j could be expressed as follows: 
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Cm = X 

g 15 
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1 1 1 

1 1 1 

1 1 1 

1 1 1 



(2) 
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Alternatively, it is possible to use a weighted average filtering method. A preferred coefficient set which yields 
better averaging results is 
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This filter can be implemented conveniently by first convolving V s j with a 4 pixel x 4 pixel filter with unity coefficients, 
then convolving the output of that filter using the same 4 pixel x 4 pixel filter. 

The output Aj jof the averaging filter 112 can be stored in data buffer 114 (Fig. 3). 

Next, the average image Aj j is processed by text detector 116 (Fig. 3) where text and low frequency half-tone 
tinting are discriminated from high frequency half-tone tinting in step 40 (Fig. 2). The text detector 116 can discriminate 



between text and half-tone tinting by several different methods. I n : ohe- method, :ar^ pixel values is 

determined, foreach piX(§iiocatiohi by subtracting the minimum Icfcal average pixel value from the maximuVn rjjSjS [fftffi- 
value in a "range rieighb^cxDd" surrbuiid the pixel. A range value R^j is l ds^&MW^BK ! 0S(^^^in : ^Q' : 
compared with a suitable predetermined threshold value T, . If the local range value exceeds the threshold value, which 
indicates that there is significant variation in the local average pixel values in the neighborhood of the pixel of interest, 
then a discrimination value D Lj is set to 1 to indicate the presence of a text pixel; otherwise, is set to 0 to indicate 
that the pixel is located in a half-tone tint region, i.e. a region where there is little variation in the local average pixel 
values. 

With regard to the size of the neighborhood over which the range value R| j is determined, a simple 3 pixel x 3 pixel 
neighborhood has been found to work well. Other neighborhood configurations are, however, possible. For example, 
the neighborhood for determining the range of the local average pixel values could consist of the pixel of interest and 
the four neighboring pixels in the vertical and horizontal directions; alternatively, it could consist of the pixel of interest 
and the four neighboring pixels in the diagonal directions. A larger range neighborhood could, of course, be used. If 
too large of a neighborhood is used, however, the determination as to whether a given pixel is a text pixel or a half- 
tone pixel will be influenced improperly by pixels that are too far removed from the given pixel to be relevant. Moreover, 
computing time and cost will increase. 

With regard to the threshold T 1t a value of 40 has been determined empirically to work well for an 8-bit represen- 
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tation scheme where 0 indicates that a pixel is completely black and 255 indicates that a pixel is completely white. The 
actual threshold value can, of course, vary depending on the particular parameters of the processing apparatus being 
used. It should be kept in mind, however, that if the threshold value is set too high, text pixels will not be detected by 
the text detector 1 1 6; if the threshold value is set too low, all half-tone pixels will improperly be classified as text pixels. 
5 Alternatively, a second detection method applies edge detection filters to the average image Aj j. For example, one 

could use a Sobel edge detector (see, for example, Pratt, "Digital Image Processing," 2nd Edition, John Wiley & Son, 
New York, 1991, p.501). In that case, one would compute the horizontal and vertical edge gradients as follows: 



10 



G H = 14 X [(A M >J+1 +2 A |J+1 + A M H ) - 
(A MH+ 2A LH+ A |+1H )] 



(4) 



is 



Gv = l4 Xl( A i + i.i*i +2A » + i.i + A ki.M)- 

( A M , j+1 + 2 A U , j + A M , j-1 H 



(5) 



If the sum of the absolute values of the two gradients exceeds a predetermined threshold T 2 . then the pixel is 
20 classified as a text pixel and the local discrimination value Dy is set to i ; otherwise, Djj is set to 0 to indicate a half- 
tone pixel. 

As a third alternative, text or lines within half-tone tinted regions can be detected even more accurately by calcu- 
lating approximations of the horizontal and vertical second derivatives of the local average i pixel values ais follows: 

L H = !4 X [(A M >H + 2A jj+1 + A j+1< K1 ) + 

( A M.H + ^H +A M.M>-^UJ (6) 



30 



L v - 1 4 X [Aj +1 +2A k1 | + A kl j. n ) + 
( A M.i + i +2A M,i + \i,H>- 4A U ] 



(7) 



If either L+, or L v exceeds a predetermined threshold T3, then the discrimination value Dj j is set to 1 to indicate a 
text pixel; otherwise it is set to 0. This detection scheme is useful in detecting 'ridges" in lines, e.g. local maxima or 
minima in horizontally oriented lines or "elbows' in vertically oriented lines. 

The discrimination values Dj j can be stored in data buffer 118. 

Video processor 120 (Fig. 3) then processes either the video image data V L j or de-screened image data Vjj in 
processing step 50 (Fig. 2) based on the discrimination values Dj j and the frequency information C0j|. De-screening, 
which is implemented by filter 121 (Fig. 3) according to methods known in the art (e.g. US-A-4, 194,221 and US-A- 
5,293,430, cited above) is necessary in order to prevent moire patterns which can result when an image already con- 
taining screen effects is screened again. Pixels for which the discrimination value Dj j is 0 are high frequency half-tone 
tint pixels, and they are rendered as such by re-screening the de-screened image V f j using a screen appropriate for 
the printer 122. 

Pixels for which the discrimination value D s j is 1 , on the other hand, are only presumptively text pixels. This is 
because the text detector 116 tends to produce a positive response (D|j=1) for pixels located in tow frequency half- 
tone areas when the averaging process uses a small averaging window and does not adequately smooth out the half- 
tone region. The frequency analyzer 109 is, however, sufficiently able to distinguish between high frequency and low 
frequency half-tone regions. Therefore, the video processor 120 processes the image data based not only on the 
discrimination signal D t y but also in conjunction with the frequency information Oj j that is output by the frequency 
detector 109. Thus, if the discrimination value Dj j is 1 and the frequency information coj j indicates that a pixel is in a 
high frequency half-tone region, the pixel can be rendered as if it is a text pixel. In that case, the originally input video 
image data V Lj is processed using a thresholding or error diffusion technique. 

Finally, the output Ojj of the video processor controls printer 122 which prints the output copy 124 (Fig. 3) in print 
step 60 (Fig. 2). Because the text pixels are rendered by a method that is most appropriate for text, while the half-tone 
tint pixels are rendered by a method most appropriate for those types of pixels, the overall appearance of the reproduced 
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compound image (Fig. 5) is significantly sharper than the appearance of the image produced without discriminating 
between half-tone tint pixels and text pixels embedded therein (Fig. 1). 

While this invention has been described in conjunction with specific embodiments thereof, it is evident that many 
alternatives, modifications, and variations will be apparent to those skilled in the art. For example, although the invention 
5 is described in the context of a digital copier, the method of the invention could be carried out on a personal computer 
using image data that is received from some auxiliary source and a computer printer. Moreover, although in the copying 
context the entire document image is customarily printed, the method of the invention could be used to ■extract* and 
print just the text portions of the document. 

10 

Claims 

1 . A method for reproducing a compound document image containing a half-tone tint region with text and/or graphics 
embedded in said half-tone tint region, said method comprising: 

15 

(a) providing a digital image of said document, said digital image comprising a multiplicity of pixels, each of 
which has a value to represent the tonal state of said pixel; 

(b) determining an average image of said document by calculating, for each said pixel, an average of the 
values of the pixels within an averaging window encompassing said pixel; 

20 (c) determining a discrimination value for each said pixel indicating whether said pixel is a half-tone tint pixel 

or a text/graphics pixel; and 

(d) controlling a print device in accordance with said discrimination values to output a reproduction of at least 
a portion of said document image. 

25 2. The method of claim 1 , wherein step (b) comprises 

(b1) calculating for each said pixel a simple average of the values of the pixels within an averaging window 
encompassing said pixel, or 

(bV) calculating for each said pixel a weighted average of the values of the pixels within an averaging window 
30 encompassing said pixel. 

3. The method of claim t or 2, wherein said averaging window has predetermined dimensions. 

4. The method of any of claims 1 to 3, wherein said averaging window has variable dimensions and said method 
35 further comprises 

(e) determining said dimensions based on a scanning resolution used to generate said compound document 
image, or 

(e*) determining said dimensions based on a frequency component of said half-tone tint region. 

40 

5. The method of any of claims 1 to 4, wherein step (c) comprises determining a discrimination value based on said 
average image. 

6. The method of claim 5, further comprising 

45 (f) determining a range value for each said pixel by calculating a range of the average values of the pixels 

in a neighborhood encompassing said pixel; and 

wherein step (c) comprises determining whether said pixel is a half-tone tint pixel or a text/graphics 
pixel based on said range value. 

50 7. The method of claim 5, further comprising 

(0 determining a gradient value for each said pixel by calculating a horizontal and a vertical edge gradient 
based on said average image; and 

wherein step (c) comprises determining whether said pixel is a half-tone tint pixel or a text/graphics pixel based 
ss on said gradient value. 

8. The method of claim 5, further comprising 
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(f") determining a second derivative value for each said pixel by calculating an approximation to a horizontal 
and a vertical second derivative of the local averages of said pixels; and 

wherein step (c) comprises determining for each said pixel whether said pixel is a half-tone tint pixel or a text/ 
graphics pixel based on said derivative value. 

5 

9. A programmable image processing apparatus when suitably programmed for carrying out the method of any of 
claims 1 to 8. 

10. An apparatus for digitally reproducing a compound document image containing a half-tone tint region with text 
10 and/or graphics embedded in said half-tone tint region, said apparatus comprising: 

an input device configured to provide a digital image of said document, said digital image comprising a mul- 
tiplicity of pixels, each of which has a value to represent the tonal state of said pixel; 
an averaging device which determines an average image of said document by calculating, for each said pixel, 
is an average of the values of the pixels within an averaging window encompassing said pixel; 

a text detector which determines a discrimination value for each said pixel indicating whether said pixel is a 
half-tone pixel or a text/graphics pixel; and 

a video processor which controls a print device in accordance with said discrimination values to output a 
reproduction of at least a portion of said document image. 

20 
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(57) A method and apparatus improves digital re- 
production of a compound document image containing 
half-tone tint regions and text and/or graphics embed- 
ded within the half-tone tint regions. The method entails 
determining (30) a local average pixel valuefor each pix- 
el in the image, then discriminating and classifying (40) 
based on the local average pixel values, text-graphics 
pixels from half-tone tint pixels. Discrimination can be 
effected by calculating a range of local averages within 
a neighborhood surrounding each pixel; by calculating 
edge gradients based on the local average pixel values; 
or by approximating second derivatives of the local av- 
erage pixel values based on the local averages. Text/ 
graphics pixels are rendered using a rendering method 
appropriate for that type of pixel; half -tone tint pixels are 
rendered using a rendering method appropriate for that 
type of pixel. 
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